Section: New Results

Approximation of Infinite Horizon Discounted Cost Markov Decision Processes

Participant : François Dufour.

Markov decision processes, infinite horizon discounted cost criterion, approximation and discretization

In this work, we deal with a discrete-time infinite horizon Markov decision process with locally compact Borel state and action spaces, and possibly unbounded cost function. Based on Lipschitz continuity of the elements of the control model, we propose a state and action discretization procedure for approximating the optimal value function and an optimal policy of the original control model. We provide explicit bounds on the approximation errors.

These results have been obtained in collaboration with Tomas Prieto-Rumeau, Department of Statistics and Operations Research, UNED, Madrid, Spain.

It has been published in the book Optimization, Control, and Applications of Stochastic Systems. In Honor of Onésimo Hernandez-Lerma [52] .